Efficient two write WOM codes, coding methods and devices

ABSTRACT

The invention provides a family of 2-write WOM-codes, preferred embodiments of which provide improved WOM-rates. Embodiments of the invention provide constructs for linear codes C having a 2-write WOM-code. Embodiments of the invention provide 2-write WOM-codes that improve the best known WOM-rates known to the present inventors at the time of filing with two writes. Preferred WOM-codes are proved to be capacity achieving when the parity check matrix of the linear code C is chosen uniformly at random. Preferred embodiments of the invention provide an electronic device utilizing an efficient coding scheme of WOM-codes with two write capability. The coding method is based on linear binary codes and allows the electronic device to write information to the memory twice before erasing it. This method can be applied for any kind of memory systems, and in particular for flash memories. The method is shown to outperform all well-known codes.

PRIORITY CLAIM AND REFERENCE TO RELATED APPLICATION

The application claims priority under 35 U.S.C. §119 and all applicabletreaties and statutes from prior provisional application Ser. No.61/353,419, which was filed Jun. 10, 2010, and is incorporated byreference herein.

FIELD

A field of the invention is data coding and compression. Embodiments ofthe invention provide WOM (Write Once Memory) coding methods anddevices.

BACKGROUND

A Write Once Memory (WOM) is a storage medium with binary memoryelements, called cells, that can change from the zero state to the onestate only once, except, in some types of memory, upon a block erase.WOM-codes were originally designed for memories that consist of binarymemory elements that could physically only be changed from a zero stateto a one state. Examples of such memories are punch cards and opticaldisks. More recently, WOM-codes have been designed for general usage indifferent types of memories, including flash memories. See, e.g., A.Jiang, “On the Generalization of Error-Correcting WOM-codes,” in Proc.IEEE Int. Symp. Inform. Theory, pp. 1391-1395, Nice, France (2007); A.Jiang and J. Bruck, “Joint coding for flash memory storage,” in Proc.IEEE Int. Symp. Inform. Theory, pp. 1741-1745, Toronto, Canada, (July2008); H. Mandavifar, P. H. Siegel, A. Vardy, J. K. Wolf, and E.Yaakobi, “A Nearly Optimal Construction of Flash Codes,” in Proc. IEEEInt. Symp. Inform. Theory. pp. 1239-1243, Seoul, Korea, (July 2009).

A WOM-code allows the reuse of a write-once medium by introducingredundancy into the recorded bit sequence and, in subsequent writeoperations, observing the state of the medium before determining how toupdate the contents of the memory with a new bit sequence.

A simple example enables the recording of two bits of information in 3memory elements, twice. The encoding and decoding rules for thisWOM-code are described in a tabular form in the table below. It is easyto verify that after the first 2-bit data vector is encoded into a 3-bitcodeword, if the second 2-bit data vector is different from the first,the 3-bit codeword into which it is encoded does not change any code bit1 into a code bit 0, ensuring that it can be recorded in the write-oncemedium.

Data Bits First Write Second Write 00 000 111 10 100 011 01 010 101 11001 110

The sum-rate of the WOM-code is the sum of all the individual rates foreach write. While there are different ways to analyze the efficiency ofWOM-codes, we find that the appropriate figure of merit is to analyzethe sum-rate under the assumption of a fixed number of writes. Ingeneral, the more writes the WOM-code can support, the better thesum-rate it can achieve. The goal is to give upper and lower bounds onthe sum-rates of WOM-codes while fixing the number of writes t to adesired number.

SUMMARY OF THE INVENTION

An embodiment of the invention provides a family of 2-write WOM-codes,preferred embodiments of which provide improved WOM-rates. Embodimentsof the invention provide constructs for linear codes C having a 2-writeWOM-code. Embodiments of the invention provide 2-write WOM-codes thatimprove the best known WOM-rates known to the present inventors at thetime of filing with two writes. Preferred WOM-codes are proved to becapacity achieving when the parity check matrix of the linear code C ischosen uniformly at random.

Preferred embodiments of the invention provide an electronic deviceutilizing an efficient coding scheme of WOM-codes with two writecapability. The coding method is based on linear binary codes and allowsthe electronic device to write information to the memory twice beforeerasing it. This method can be applied for any kind of memory systems,and in particular for flash memories. The method is shown to outperformall well-known codes.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 plots computer search determined rates and theoretical capacityfor prior WOM-codes;

FIG. 2 illustrates a Blackwell channel;

FIG. 3 plots computer search determined rates and theoretical capacityof the Blackwell channel;

FIG. 4 illustrates a [3,3:4,3,2] three-write WOM-code.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention addresses two problems related to 2-write WOM-codes 1) Thenumber of messages written to the memory on each write is the same; 2)Different number of messages can be written on each write. For the caseof 2-write WOM-codes, the theoretical bound on the WOM-rate for thefirst problem is approximately 1.5458 and in the second problem it isapproximately 1.58. Since the best known WOM-rate for the first problemis approximately 1.34 and 1.37 for the second problem, there is stillroom for improvement in closing these gaps. The invention provides afamily of 2-write WOM-codes, preferred embodiments of which provideimproved WOM-rates. Embodiments of the invention provide constructs forlinear codes C having a 2-write WOM-code. Embodiments of the inventionprovide 2-write WOM-codes that improve the best known WOM-rates known tothe present inventors at the time of filing with two writes. PreferredWOM-codes are proved to be capacity achieving when the parity checkmatrix of the linear code C is chosen uniformly at random.

Preferred embodiments of the invention provide an electronic deviceutilizing an efficient coding scheme of WOM-codes with two writecapability. The coding method is based on linear binary codes and allowsthe electronic device to write information to the memory twice beforeerasing it. This method can be applied for any kind of memory systems,and in particular for flash memories. The method is shown to outperformall well-known codes.

Preferred embodiments of the invention are applicable to memories havingcells that can change their state from zero to one but not from one tozero except upon an erase of the entire memory. Preferred embodiments ofthe invention are t-write WOM-codes that conform to Thoerem 1 in thedescription below.

Preferred embodiments of the invention will now be discussed withrespect to the drawings. The drawings may include schematicrepresentations, which will be understood by artisans in view of thegeneral knowledge in the art and the description that follows. Featuresmay be exaggerated in the drawings for emphasis, and features may not beto scale.

Two-Write WOM-Codes

Preferred embodiment methods and devices use a two-write WOM-codesconstruction that reduces the gap between the upper and lower bound onthe sum-rates for both fixed- and unrestricted-rate WOM-codes. InReference [28], a “coset-coding” is used only on the second write inorder to generate an ε-error two-write WOM-codes. However, in ε-errortwo-write WOM-codes, the second write is not guaranteed in the worstcase but is allowed with high probability. Methods and codes of theinvention guarantee from every linear code a two-write WOM-code. A“coset-coding” scheme only on the second write is used as in Reference[28], but the first write is modified such that the second write isguaranteed in the worst case. Preferred specific embodiment WOM-codeshave better sum-rates than the previously best known codes discussedabove. Preferred embodiment WOM-codes choose uniformly at random theparity-check matrix of the linear code, such that there exist WOM-codesthat achieve all points in the capacity region of two-write WOM-codes.An example application of a preferred method generate from eachtwo-write WOM-code a code for the Blackwell channel.

A. Two-Write WOM-Codes Construction

Let C[n,k] be a linear code with parity-check matrix

. For each vε{0,1}^(n) we define the matrix

_(v) as follows. The i-th column of

_(v), 1≦i≦n, is the i-th column of

if v_(i)=0 and otherwise it is the zeros column. The set Vc is definedto beVc={vε{0,1}^(n)|rank(

_(v))=n−k}.  (1)

We first note the following position. If a vector v belongs to Vc, itsweight is at most k.

The support of a binary vector v, denoted by supp(v), is the set{i/v_(i)=1}. The dual of the code C is denoted by C⊥. The next lemma isa variation of a well known result (see e.g. Reference [5]).

Lemma 1. Let C[n, k] be a linear code with parity-check matrix

. For each vector vε{0, 1}^(n), rank(

_(v))=n−k if and only if v does not cover any non-zero codeword in C⊥.

Lemma 1 implies that if two matrices are parity-check matrices of thesame linear code C, then their corresponding sets Vc are identical, andso we can define the set Vc to be

Vc={vε{0, 1}^(n)|v does not cover any non-zero cεC⊥}.

The next theorem presents the preferred embodiment two-write WOM-codes,

Theorem 1.

Let C[n,k] be a linear code with parity-check matrix

and let Vc be the set defined in (1). Then there exists an [n,2;|Vc|,2^(n−k)] two-write WOM-code of sum-rate

$\frac{{\log_{2}{V_{c}}} + \left( {n - k} \right)}{n}$

The two-write WOM-code can be proven by showing the existence of theencoding and decoding maps on the first and second writes. First, let{v₁, v₂, . . . v, |v_(c)|} be an ordering of the set Vc. The first andthe second writes are implemented as follows.

1) On the first write, a symbol over an alphabet of size |V_(c)| iswritten. The encoding and decoding maps E₁, D₁ are defined as follows.For each mε{1, . . . , |V_(c)|}, E₁ (m)=v_(m) and D₁ (v_(m))=m.

2) On the second write, we write a vector s₂ of n−k bits. Let v₁ be theprogrammed vector on the first write and s₁=

_(v1), thenE ₂(s ₂ ,v ₁)=v ₁ +v ₂,where v₂ is a solution of the equation

_(v1)·v₂=s₁+s₂. For the decoding map D₂, if c is the vector ofprogrammed cells, then the decoded value of the n−k bits is given byD₂(c)=

·c=

·v₁+

·v₂=s₁+s₁+s₂=s₂.

The success of the second write results from the condition that forevery vector vεV_(c), rank (

_(v))=n−k.

There is no condition on the code C and therefore we can use any linearcode in this construction, though we seek to find codes that maximizethe sum-rate

$\frac{{\log_{2}{V_{c}}} + \left( {n - k} \right)}{n}.$Next, we show two examples of two-write WOM-codes that achieve bettersum-rates than the previously best known ones.

Example 1

This example demonstrates how Theorem 1 works for the [16,5,8] firstorder Reed-Muller code and demonstrates a rate of 1.4566. Its dual codeis the [16,11,4] second order Reed-Muller, which is the extended Hammingcode of length 16. Hence, we are interested in the size of the set

V_(i)={vε{0,1}¹⁶| v does not cover any cε[16,11,4]}.

According to Equation (1), the set V₁ does not contain vectors of weightgreater than five. This extended Hamming code has 140 codewords ofweight four and no codewords of weight five. The set V₁ consists of thefollowing vector sets.

1) All vectors of weight at most three. There are Σ_(i=0) ³(_(i) ¹⁶)=697such vectors.

2) All vectors of weight four that are not codewords. There are (₄¹⁶)−140=1680 such vectors.

3) All vectors of weight five that do not cover a codeword of weightfour. There are (₅ ¹⁶)−12·140=2688 such vectors. Since the minimumdistance of the code is four, a vector of weight five can cover at mostone codeword of weight four.

Therefore, we get |V_(i)|=697+1680+2688=5065 and the sum-rate is(log₂(5065)+11)/16=1.4566.

It is possible to modify this WOM-code such that on the first write only11 bits are written. Thus, we achieve a two-write fixed-rate WOM-codeand its sum-rate is 22/16=1.375, which is the best known fixed-rateWOM-code.

Example 2

In this example we will use the [23,11,8] Golay code. Its dual code isthe [23,12,7] Golay code so we are interested in the size of the setV₂={vε{0, 1}²³ |v does not cover any cε[23, 12,7]}.

According to Equation (1), there are no vectors of weight greater than11 in the set V₂. The invention achieves a rate of 1.4632. The [23,12,7]Golay code has A₇=253 codewords of weight seven, A₇=506 codewords ofweight eight, and A₁₁=1288 codewords of weight 11. The set V₂ consistsof the following vector sets.

1) All vectors of weight at most 6. This number of vectors is Σ_(i=0)⁶(_(i) ²³)=145499.

2) All vectors of weight between 7 and 10 besides those that cover acodeword of weight 7 or 8. Since the minimum distance of the code is 7every vector can cover at most one codeword. Hence, this number ofvectors is

${{\sum\limits_{i = 7}^{10}\begin{pmatrix}23 \\i\end{pmatrix}} - {A_{7} \cdot {\sum\limits_{i = 7}^{10}\begin{pmatrix}16 \\{i - 7}\end{pmatrix}}} - {A_{8} \cdot {\sum\limits_{i = 8}^{10}\begin{pmatrix}15 \\{i - 8}\end{pmatrix}}}} = 2459160.$

3) All vectors of weight 11 that are not codewords and do not cover acodeword of weight either 7 or 8. This number was shown in [6] to be695520.

Therefore, for the [23,11,8] Golay code we get|v₂|=145499+2459160+695520=3300179, and thus the sum-rate is(log₂(3300179)+12)/23=1.4632.B. Random Coding

The preferred coding and coding methods consistent with FIG. 1 can beshown to work for any linear code C. Given a linear code C[n,k] withparity-check matrix H_(c), we denote

${{\mathcal{R}_{1}(\mathcal{C})} = \frac{\log_{2}{V_{C}}}{n}},{{\mathcal{R}_{2}(\mathcal{C})} = \frac{n - k}{n}}$so the sum-rate of the generated WOM-codes is

${{\mathcal{R}_{1}(\mathcal{C})} + {\mathcal{R}_{2}(\mathcal{C})}} = {\frac{{\log_{2}{V_{\mathcal{C}}}} + n - k}{n}.}$

Our goal in this subsection is to show that it is possible to achievethe capacity region C₂ of a t-write WOM-code by choosing uniformly atrandom the parity-check matrix of the linear code C. We prove that inthe following theorem.

Theorem 2.

For any (R₁, R₂)εC₂ and ε>0 there exists a linear code C satisfyingR₁(C)≧R₁−ε, R₂(C)≧R₂−ε.

Proof: Let pε[0,0.5] be such that R₁≦h(p) and R₂≦1−p. Let k=┌np┐ for nlarge enough and let us choose uniformly at random an (n−k)×n matrix H.The matrix twill be the parity-check matrix of the linear code C thatwill be used to construct the two-write WOM-code. For each vector vε{0,1}^(n), let us define the indicator random variable X_(v) (

) on the space of all matrices as follows

${X_{v}(H)} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} v} \in V_{\mathcal{C}}} \\0 & {otherwise}\end{matrix} \right.$where Vc is the set defined in Equation (1). Note that choosing thematrix

uniformly at random induces a probability distribution on the set Vc andthus a probability distribution on the random variable X_(v)(

). Then the number of vectors in Vc is X(

)=Σvε{0,1}^(n) X_(v)(

), and

$\begin{matrix}{{E\left\lbrack {X(H)} \right\rbrack}{\sum\limits_{v \in {\{{0,1}\}}^{''}}{{E\left\lbrack {X_{v}(H)} \right\rbrack}{\sum\limits_{v \in {\{{0,1}\}}^{''}}{\Pr{\left\{ {{X_{v}(H)} = 1} \right\}.}}}}}} & (2)\end{matrix}$

We maintain that Pr {Xv (H)=1} depends on v only through its weight, wt(v). In this case, (2) simplifies to

$\begin{matrix}{{E\left\lbrack {X(H)} \right\rbrack} = {\sum\limits_{i = 0}^{n}{\begin{pmatrix}n \\i\end{pmatrix}\Pr\left\{ {{X_{{v:{{wt}{(v)}}} = i}(H)} = 1} \right\}}}} \\{{= {\sum\limits_{i = 0}^{k}{\begin{pmatrix}n \\i\end{pmatrix}\Pr\left\{ {{X_{{v:{{wt}{(v)}}} = i}(H)} = 1} \right\}}}},}\end{matrix}$because if wt(v)≧k−1 then X_(v)=0 (Equation (1)).

Now, let us determine the value of Pr {X_(v) (H)=1} for a vector v ofweight 0≦i≦k. Note that vεVc if and only if the sub-matrix of size(n−k)×(n−wt (v)) induced by the zero entries of the vector v is fullrank. It is well known, that if we choose an m×n matrix, where m≦n,uniformly at random then the probability that it is full rank isΠ_(i=n−m+1) ^(n)(1−2−i). Therefore, if we choose an (n−k)×(n−i) matrixuniformly at random then the probability that it is full rank isΠ_(i=k−i+1) ^(n−i)(1−2^(−i)). Note that

${{{\prod\limits_{j = {k - i + 1}}^{n - i}\left( {1 - 2^{- j}} \right)} > {\prod\limits_{j = 1}^{\infty}\left( {1 - 2^{- j}} \right)} > {\left( {1 - \frac{1}{2}} \right)\left( {1 - {\sum\limits_{j = 2}^{\infty}2^{- j}}} \right)}} = {{\frac{1}{2} \cdot \frac{1}{2}} = \frac{1}{4}}},$and hence, Pr{X_(v)(H)=1}=Π_(i=k−i+1) ^(n−i)(1−2^(−i))>¼ According toLemma 4.8 in Reference [24],

${\sum\limits_{i = 0}^{k}\begin{pmatrix}n \\i\end{pmatrix}} \geqslant {\frac{1}{n + 1}2^{{nh}{(\frac{k}{n})}}}$and, therefore, we get

${E\left\lbrack {X(H)} \right\rbrack} = {{\sum\limits_{i = 0}^{k}{\begin{pmatrix}n \\i\end{pmatrix}{\prod\limits_{j = {k - i + 1}}^{n - i}\left( {1 - 2^{- j}} \right)}}} > {2^{{{nh}{(\frac{k}{n})}} - 2 - {\log_{2}{({n + 1})}}}.}}$

It follows that there exists a parity-check matrix

of a linear code C, such that the size of the set Vc is at least

$2^{{{nh}{(\frac{k}{n})}} - 2 - {\log_{2}{({n + 1})}}}$and

${\mathcal{R}_{1}(\mathcal{C})} \geqslant {{h\left( \frac{k}{n} \right)} - \frac{2 + {\log_{2}\left( {n + 1} \right)}}{n}} \geqslant {{h(p)} - \frac{2 + {\log_{2}\left( {n + 1} \right)}}{n}} \geqslant {\mathcal{R}_{1} - ɛ}$${\mathcal{R}_{2}(\mathcal{C})} = {\frac{n - k}{n} \geqslant {\left( {1 - p} \right) - \frac{1}{n}} \geqslant {\mathcal{R}_{2} - \varepsilon}}$for n large enough.

Random coding was proved to be capacity-achieving by constructing apartition code References [14], [9]. However, the present random codingmethod has more structure that enables to look for WOM-codes with arelatively small block length. We ran a computer search to look for suchWOM-codes. The parity-check matrix of the linear code C was chosenuniformly at random and then the size of the set Vc was computed. Theresults are shown in FIG. 1. Note that if (R₁, R₂) and (R₃, R₄) are twoachievable rate points then for each tεQ the point (t

₁+(1−t)

₂, t

₃+(1−t)

₄) is an achievable rate point, too. This can simply be done by blocksharing of a large number of blocks. Therefore, the achievable region isconvex.

We ran a computer search to find more two-write WOM-codes with highsum-rates. For fixed-rate WOM-codes, our best construction achieved by acomputer search has sum-rate

$\frac{48}{33} \approx 1.4546$and for unrestricted-rate WOM-codes our best computer searchconstruction achieved sum-rate 1.4928. The number of cells in these twoconstructions is 33.

The encoding and decoding maps of the second write are implemented bythe parity-check matrix of the linear code C as described in the proofof Theorem 1. A naive scheme to implement the encoding and decoding mapsof the first write is simply by a lookup table of the set Vc. However,this can be done more efficiently using algorithms to encode and decodeconstant weight binary codes. There are several works which efficientlyencode and decode all binary vectors of length n and weight k and can beused; see for example References [2], [7], [19], [25], [26]. These workscan be easily extended to construct efficient encoder and decoder mapsto the set of all binary vectors of length n and weight at most k,denoted byB(n,k)={vε{0,1}^(n)|supp(v)≦k}.

The set Vc is a subset of the set B(n, k). Therefore, we can use thesealgorithms while constructing a smaller table, only for the vectors inthe set B(n,k)\V_(c) as follows. Assume that ƒ: {1, . . . ,|B(n,k)|}→B(n,k) is a one-to-one and onto map such that the complexityto calculate ƒ and ƒ⁻¹ is efficient. Assume we list all the vectors inB(n,k)\V_(C) such that we list for every vector vεB(n,k)\V_(C) its valueƒ⁻¹(v) and this list is sorted according to the values of ƒ⁻¹(v). Then,a mapping g: {1, . . . , |V_(C)|}→V_(C) is constructed such that for allxε{1, . . . |V_(C)|}, g(x)=ƒ(x+a(x)), where a(x) is the number ofvectors in B(n,k)\V_(C) of value less than x. The time complexity tocalculate a(x) is a(x) is O(log₂(|B(n,k)\V_(C)|)) since this list issorted. Similarly, for all vεV_(C), g⁻¹(v)=ƒ⁻¹(v)−a(ƒ⁻¹(v)).

In many cases, the size of the set B (k,n)\Vc will be significantlysmaller than the size of Vc. For example, for the Golay code [23,11,8]the size of Vc is 3300179 while the size of B(23, 11)\Vc is

${{\sum\limits_{i = 0}^{11}\begin{pmatrix}n \\i\end{pmatrix}} - 3300179} = 894125.$Similarly, for the Reed-Muller code [16,5,8] the size of the set Vc is5065 while the size of the set B(16, 5)\Vc is 1820.C. Application to the Blackwell Channel

The Blackwell channel, introduced first by Blackwell [1], is one exampleof a deterministic broadcast channel. The channel is composed of onetransmitter and two receivers. The input to the transmitter is ternaryand the channel output to each receiver is a binary symbol. Let u be theternary input vector to the transmitter of length n. For 1≦i≦n,ƒ(u_(i))=(ƒ(u_(i))₁, ƒ(u_(i))₂), is a binary vector of length twodefined as follows (FIG. 2):ƒ(0)=(0,0), ƒ(1)=(0,1), ƒ(2)=(1,0).

The binary vectors ƒ₁(u), ƒ₂(u) are defined to beƒ₁(u)=(ƒ(u ₁)₁, ƒ(u ₂)₁, . . . , ƒ(u _(n))₁),ƒ₂(u)=(ƒ(u ₁)₂, ƒ(u ₂)₂, . . . , ƒ(u _(n))₂),and are the output vectors to the two receivers.

The capacity region of the Blackwell channel was found by Gel'fand [11]and consists of five sub-regions, given by their boundaries:{(R ₁ ,R ₂)|0≦R ₁<½, R ₂=1},  1){(R ₁ ,R ₂)|R ₁=1−p, R ₂ =h(p), ⅓≦p≦½},  2){(R ₁ ,R ₂)|R ₁R₂=log₂ 3, ⅔≦R ₁≦log₂3−⅔},  3){(R ₁ ,R ₂)|R ₁ =h(p), R ₂=1−p, ⅓≦p≦½},  4){(R ₁ ,R ₂)|R ₁=1,0≦R ₂≦½}.  5)

The connection between the Blackwell channel and two-write WOM-codes wasidentified by Roth [23]. The next theorem shows that from everytwo-write WOM-code of rate (R₁, R₂) it is possible to construct codesfor the Blackwell channel of rates (R₁,R₂) and (R₂,R₁).

Theorem 3.

If (R₁,R₂) is an achievable rate of a two-write WOM-code, then (R₁, R₂)and (R₂, R₁) are achievable rates on the Blackwell channel. Proof Assumethat there exists a [n, 2; 2^(nR) ¹ , 2^(nR) ² ] two-write WOM-code andlet E₁, E₂ and D₁, D₂ be its encoding and decoding maps. We maintainthat there exists a coding scheme for the Blackwell channel of rate (R₁,R₂). Let (m₁, m₂)ε{1, . . . , 2^(nR1)}×{1, . . . , 2^(nR2)} be twomessages and let v₁=E₁(m₁) and v₂=E₂(m₂,v₁). Let u be a ternary vectorof length n defined as follows. For 1≦i≦n, u_(i)=ƒ⁻¹(v_(1,i), v_(2,i) ).The vector u is well-defined since for all 1≦i≦n, (v_(1,i),v_(2,i))≠(1,0) and hence (v_(1,i), v_(2,i) )≠(1,1). The vector u is theinput to the transmitter. Then, the vector ƒ₁(u) is transmitted to thefirst receiver and the vector ƒ₂(u) to the second receiver. Note thatƒ₁(u)=v₁ and ƒ₂(u)= v₂ . Therefore, the first receiver decodes itsmessage according to

₁(ƒ₁(u))=

₁(v₁)=m₁ and the second receiver decodes its message according to

₂( ƒ₂(u))=

₂(v₂)=m₂.

Similarly, it is possible to achieve the rate (R₂, R₁). Now we letv₂=E₂(m₂) and v₁=E₁ (m₁, v₂). The vector u is defined as u_(i)=ƒ⁻¹(v_(1,i) , v_(2,i)) for 1≦i≦n. The decoded message by the first receiveris D₁(ƒ₁(u)) and D₂(ƒ₂(u)) is the decoded message by the secondreceiver.

It is possible to define the Blackwell channel differently such that theforbidden pair of bits is not (1, 1) but another combination. Ourconstruction of the codes can be adjusted accordingly.

Now, we can use our two-write WOM-codes in order to define codes for theBlackwell channel. By using time sharing, the achievable region isconvex and hence we get in FIG. 3 the capacity and achieved regions forthe Blackwell channel.

Multiple-Write WOM-Codes

The invention also provides WOM-code constructions which reduce the gapsbetween the upper and lower bounds on the sum-rates of WOM-codes for3≦t≦10. First, we generalize the two-write WOM-code construction fromabove for non-binary cells. Then, we show how to use these non-binarytwo-write WOM-codes in order to construct binary multiple-writeWOM-codes. We start with specific constructions for three- andfour-write WOM-codes, and then show a general design approach that worksfor an arbitrary number of writes.

A. Non-Binary Two-Write WOM-Codes

Suppose now that each cell has q levels, where q is a prime number or apower of a prime number. We start by choosing a linear code C[n,k] overGF(q) with a parity-check matrix

of size (n−k)×n. For a vector v of length n over GF(q), let

(v) be the matrix

with zero columns replacing the columns that correspond to the positionsof the non-zero values in v. Then we defineV _(c) ^((q)) ={vε(GF(q))^(n)|rank(H(v))=n−k}.  (3)

Next, we construct a non-binary two-write WOM-code [n,2; |V_(c) ^((q))|,q^(n−k)] in a similar manner to the construction in Section IV. Sincethe proof of the next theorem is very similar to the proof of Theorem 4we omit it. A complete proof can be found in [18].

Theorem 4.

Let C[n,k] be a linear code with parity-check matrix

over GF(q) and let V_(c) ^((q)) be the set defined in (3). Then thereexists a q-ary [n,2; |V_(c) ^((q))|, q^(n−k)] two-write WOM-code ofsum-rate

$\frac{{\log_{2}{V_{c}^{(q)}}} + {\left( {n - k} \right)\log_{2}q}}{n}.$As was shown in the binary case, there is no restriction on the choiceof the linear code C or the parity-check matrix

. Every such code/matrix generates a WOM-code. For a linear code C wedefine

${\mathcal{R}_{1}(\mathcal{C})} = {{\frac{\log_{2}{V_{\mathcal{C}}^{(q)}}}{n}\mspace{14mu}{and}\mspace{14mu}{\mathcal{R}_{2}(\mathcal{C})}} = \frac{\left( {n - k} \right)\log_{2}q}{n}}$so the sum-rate of the generated WOM-code is R₁ (C)+R₂ (C). The capacityregion of the achievable rates by this construction is

$C_{2}^{(q)} = {\left\{ {{\left( {\mathcal{R}_{1},\mathcal{R}_{2}} \right)❘{\exists{p \in \left\lbrack {0,\frac{q - 1}{q}} \right\rbrack}}},{\mathcal{R}_{1} \leqslant {{h(p)} + {p\;{\log_{2}\left( {q - 1} \right)}}}},{\mathcal{R}_{2} \leqslant {\left( {1 - p} \right){\log_{2}(q)}}}} \right\}.}$

Theorem 5.

For any

₁,

′₂) εC₂ ^((q)) and ε>0, there exists a linear code C satisfyingR₁(C)≧R₁−ε, R₂(C)≧R₂−ε.

The next corollary provides the best achievable sum-rate of theconstruction.

Corollary.

For any q-ary WOM-code generated using our construction, the highestachievable sum-rate is log₂ (2q−1).

Proof: First, note that

${{{h(p)} + {p\;{\log_{2}\left( {q - 1} \right)}} + {\left( {1 - p} \right)\log_{2}q}} = {{p\;{\log_{2}\left( \frac{q - 1}{p} \right)}} + {\left( {1 - p} \right){\log_{2}\left( \frac{q}{1 - p} \right)}}}},$and since the function ƒ(x)=log₂ x is a concave function

${{{p\;\log_{2}\frac{q - 1}{p}} + {\left( {1 - p} \right)\log_{2}\frac{q}{1 - p}}} \leqslant {\log_{2}\left( {{p \cdot \frac{q - 1}{p}} + {\left( {1 - p} \right)\frac{q}{1 - p}}} \right)}} = {{\log_{2}\left( {{2q} - 1} \right)}.}$

Also, for

${p = \frac{q - 1}{{2q} - 1}},$the achievable sum-rate is log₂(2q−1). Therefore, there exists aWOM-code produced by our construction with sum-rate log₂ (2q−1).

On the other hand, any WOM-code resulting from our constructionsatisfies the property that every cell is programmed at most once. Thismodel was studied in Reference [9] and the maximum achievable sum-ratewas proved to be log₂ (2q−1). Therefore, our construction cannot producea WOM-code with a sum-rate that exceeds log₂(2q−1).

This construction does not achieve high sum-rates for non-binarytwo-write WOM-codes in general. While the best achievable sum-rate ofthe construction is log₂ (2q−1), the upper bound on the sum-rate islog₂(2q−1). The decrease in the sum-rate in our construction resultsfrom the fact that cells cannot be programmed twice. That is, if a cellwas programmed on the first write, it cannot be reprogrammed on thesecond write even if it did not reach its highest level. In fact, it ispossible to find non-binary two-write WOM-codes with better sum-rates.However, the goal is not to find efficient non-binary WOM-codes. Rather,the non-binary codes that we have constructed can be used in the designof binary multiple-write WOM-codes.

For the construction of binary multiple-write, we use WOM-codes overGF(3). We ran a computer search to find such a ternary two-writeWOM-code of sum-rate 2.2205, and we will use this WOM-code in order toconstruct specific multiple-write WOM-codes.

B. Three-Write WOM-Codes

We start with a construction for binary three-write WOM-codes. Theconstruction uses the WOM-codes found in the previous subsection overGF(3).

Theorem 6.

Let C₃ be an [n, 2; ^(n)

, 2^(n)

²] two-write WOM-code over GF(3) constructed as above in Section A.Then, there exists a [2n, 3:2^(n)

2^(n)

², 2^(n)] three-write WOM-code of sum-rate

$\frac{R_{1} + R_{2} + 1}{2}.$

Proof: We denote by E_(3,1) and E_(3,2) the encoding maps of the firstand second writes, and by D_(3,1) and D_(3,2) the decoding maps of thefirst and second writes of the WOM-code C₃, respectively. The 2n cellsof the three-write WOM-code we construct are divided into n two-cellblocks, so the memory-state vector is of the form ((c_(1,1), c_(1,2)),c_(2,1), c_(2/2)), . . . , (c_(n,1), c_(n,2))). In this construction wealso use map φ: GF(3)

(GF(2), GF2)) defined as follows:φ(0)=(0,0),φ(1)=(1,0),φ(2)=(0,1),

The map φ extends naturally to ternary vectors v=(v₁, . . . ,v_(n))εGF(3)^(n) using the ruleφ(v)=(φ(v ₁), . . . , φ(v _(n))),

On the pairs (c,c′) in the image of ø, we define ø⁻¹(c,c′) to indicatethe inverse function. The map ø⁻¹ is extended similarly to work overvectors of such bit pairs. We are now ready to describe the encoding anddecoding maps for a three-write WOM-code.

1) On the first write, a message in from the set {1, . . . , 2^(n)

¹} is written in the 2n cells:ε₁(m)=φ(ε_(3,1)(m)).

The decoding map is defined similarly, where c is the memory-statevector:

₁(c)=

_(3,1)(φ⁻¹(c)).

2) On the second write, a message in from the set {1, . . . , 2^(n)

²} is written in the 2n cells as follows. Let c be the programmed vectoron the first write. Then,ε₂(m,c)=φ(ε_(3,2)(m,φ ⁻¹(c)).

That is, first the memory-state vector c is converted to a ternaryvector. Then, it is encoded using the encoding E_(3,2) and the newmessage, producing a new ternary memory-state vector. Finally, the lastvector is converted to a 2n-bit vector. The decoding map is defined ason the first write:

₂(c)=

_(3,2)(φ⁻¹(c)).

According to the construction of the WOM-code C₃, no ternary cell isprogrammed twice and therefore each of the n pairs of bits is programmedat most once.

3) On the third write, an n-bit vector v is written. Let c=((c_(1,1),c_(1,2)), . . . , (c_(n,1), c_(n,2))) be the current memory-statevector. Then,E ₃(v,c)=((c′ _(1,1) ,c′ _(1,2)), . . . ,(c′ _(n,1) ,c′ _(n,2)))is a vector, defined as follows. For 1≦i≦n, Yc′ _(i,1), c′_(i,2))=(1,1)if v_(i)=1 and otherwise (c′_(i,1), c′_(i,2)). It is always possible toprogram the pair of bits to be (1, 1) since at most one cell in eachpair was previously programmed. The decoding map D₂(c) is defined to beD ₂(c)=(c _(1,1) ·c _(1,2) , . . . ,c _(n,1) ·c _(n,2)).

That is, the decoded value of each pair of bits is one if and only ifthe value of both of them is one.

Corollary.

The best achievable sum-rate of a three-write WOM-code using thisconstruction is (log₂ 5+1)/2≈1.66.

Proof: Given a two-write WOM-code C₃ over GF(3) with rates (R₁, R₂), theconstructed binary three-write WOM-code has rates (R₁/2, R₂/2, ½) andits sum-rate is R=(R₁+R₂+1)/2. This sum-rate is maximized when R₁+R₂ ismaximized. But R₁+R₂ is the sum-rate of the two-write WOM-code overGF(3), which was proven in Corollary 9 to be maximized at log₂ 5. Thenthe maximum achievable sum-rate of the constructed binary three-writeWOM-code is

$\frac{{\log_{2}5} + 1}{2} \approx {1.66.}$

Using the construction of WOM-codes over GF(3) presented above, we canconstruct a three-write WOM-code of sum-rate (2.2205+1)/2=1.6102.

C. Four-Write WOM-Codes

We next present a construction for four-write binary WOM-codes.

Theorem 7.

Let C₃ be an [n,2;2^(nR) _(3,1),2^(nR) _(3,2)] two-write WOM-code overEquation (2) constructed as above. Let C₂ be an [n,2;2^(nR) _(2,1),2^(nR) _(2,2)] binary two-write WOM-code. Then, there exists a[2n,4;2^(nR) _(3,1), 2^(nR) _(3,2), 2^(nR) _(2,1)2^(nR) _(2,2)]four-write WOM-code of sum-rate

$\frac{R_{3.1} + R_{3.2} + R_{2.1} + R_{2.2}}{2}.$

Proof: The proof is very similar to the one used for three-writeWOM-codes. We denote by E_(3,1), E_(3,2) the encoding maps of the firstand second writes, and by D_(3,1), D_(3,2) the decoding maps of thefirst and second writes of the WOM-code C₃, respectively. Similarly, theencoding and decoding maps of the WOM-code C₂ for the first and secondwrites are denoted by E_(3,1), E_(3,2) and D_(3,1), D_(3,2),respectively. Using the encoding and decoding maps of C₃, we define thefirst and second writes of our constructed four-write WOM-code as we didfor the first and second writes of the three-write WOM-codes. The thirdand fourth writes are defined in a similar way, as follows.

1) On the third write, a message m from the set{1, . . . ,2^(nR) ^(2,1) } is written. Let (ξ_(2,1)(m)=v=(v ₁ , . . . ,v_(n)) and let c=((C _(1,1) ,C _(1,2)), . . . ,(c _(n,1) ,c _(n,2)))

be the current memory-state vector. Then,ε₃(m,c)=((c′ _(1,1) ,c′ _(1,2)), . . . ,(c′ _(n,1) ,c′ _(n,2))),

where for 1≦i≦n, (c′_(i,1), c′_(i,2))=(1,1) if v_(i)=1 and, otherwise,(c′_(1,1), c′_(1,2))=(c_(i,1), c_(i,2)). The decoding map D_(3(c)) isdefined to beD ₃(c)=D _(2,1)(c _(1,1) ·c _(1,2), . . . ,c_(n,1) ·c _(n,2)).

2) On the fourth write, a message m from the set

{1, . . . , 2^(n)

^(2,2)} is written. Letε_(2,2)(m,(c _(1,1) ·c _(1,2) , . . . ,c _(n,1) ·c _(n,2)))=v=(v ₁ , . .. ,v _(n)),

where c=((c_(1,1), c_(1,2)), . . . , (c_(n,1), c_(n,2))) is the currentmemory-state vector. Then,ε₄(m,c)=((c _(1,1) ′,c _(1,2)′), . . . ,(c _(n,1) ′,c _(n,2)′)),

where for 1≦i≦n, (c′_(1,1), c′_(1,2))=(1,1) if v_(i)=1 and, otherwise,(c_(i,1)′, c_(i,2)′)=(c_(i,1), c_(i,2)). The decoding map D₄(c) isdefined, as before, by

₄(c)=

_(2,2)(c _(1,1) ′·c _(1,2) ′, . . . ,c _(n,1) ′·c _(n,2)′).

The last theorem requires both the binary two-write and ternarytwo-write WOM-codes to have the same number of cells, n. However, we canconstruct a four-write binary WOM-code using any two such WOM-codes,even if they do not have the same number of cells. Suppose we have aWOM-code over GF(3) with n₁ cells and binary WOM-code with n₂ cells.Both codes can be extended to use (n₁,n₂) cells. Then, the constructionabove will give a four-write WOM-code.

Corollary.

The best achievable sum-rate of a four-write WOM-code using thisconstruction is (log₂5+log₂3)/2≈1.95.

Proof: The maximum value of R_(3,1)+R_(3,2) is log₂ 5 and the maximumvalue of R_(2,1)+R_(2,2) is log₂ 3. Therefore, the maximum sum-rate ofthe constructed

$\frac{{\log_{2}(5)} + {\log_{2}(3)}}{2} \approx {1.95.}$

If we use the WOM-code over GF(3) of sum-rate 2.2205 found in theprevious subsection as the WOM-code C₃ and the binary two-write WOM-codeof sum-rate 1.4928 found as the WOM-code C₂, then there exists afour-write WOM-code of sum-rate (2.2205+1.4928)/2=1.8566.

C. Multiple-Write WOM-Codes

The construction of three- and four-write WOM-codes can be easilygeneralized to an arbitrary number of writes. We state the followingtheorem and skip its proof since it is very similar to the proofs of thecorresponding theorems for three- and four-write WOM-codes.

Theorem 8.

Let C₃ be an [n,2; 2^(n)

^(3,1), 2^(n)

^(3,2)] two-write WOM-code over GF(3) constructed as above. Let C₂ be an[n,t−2; 2^(n)

^(2,1), . . . , 2^(n)

^(2i−2)] binary (t−2)-write WOM-code. Then, there exists a[2n,t;2^(nR) _(3,1),2^(nR) _(3,2),2^(nR) _(2,1), . . . ,2^(nR) _(2t−2)]t-write WOM-code of sum-rate

$\frac{\mathcal{R}_{3,1} + \mathcal{R}_{3,2} + {\sum\limits_{i = 1}^{t - 2}\mathcal{R}_{2,i}}}{2}.$Theorem 14 implies that if there exists a (t−2)-write WOM-code ofsum-rate R_(t−2) then there exists a t-write WOM-code of sum-rate

$R_{t} = {\frac{{\log_{2}5} + R_{t - 2}}{2}.}$The following corollary summarizes the possible achievable sum-rates oft-write WOM-codes.

Corollary.

For t≧3, there exists a t-write WOM-code of sum-rate

$\mathcal{R}_{t} = \left\{ \begin{matrix}{\frac{{{\left( {2^{\frac{t - 1}{2}} - 1} \right) \cdot \log_{2}}5} + 1}{2^{\frac{t - 1}{2}}},} & {t\mspace{14mu}{odd}} \\{\frac{{{\left( {2^{\frac{t - 2}{2}} - 1} \right) \cdot \log_{2}}5} + {\log_{2}3}}{2^{\frac{t - 2}{2}}},} & {t\mspace{14mu}{{even}.}}\end{matrix} \right.$If we use again the two-write WOM-code over GF(3) of sum-rate 2.2205 andthe binary two-write WOM-code of sum-rate 1.4928 from Section IV, thenfor t≧3 we obtain a t-write WOM-code of sum-rate R_(t′) where

$\mathcal{R}_{t} = \left\{ \begin{matrix}{\frac{{\left( {2^{\frac{t - 1}{2}} - 1} \right) \cdot 2.22} + 1}{2^{\frac{t - 1}{2}}},} & {t\mspace{14mu}{odd}} \\{\frac{{\left( {2^{\frac{t - 2}{2}} - 1} \right) \cdot 2.22} + 1.4928}{2^{\frac{t - 2}{2}}},} & {t\mspace{14mu}{{even}.}}\end{matrix} \right.$

Concatenated WOM-Codes

The construction presented in the previous section provides us with afamily of WOM-codes for all t≧3. In this section, we will show a generalscheme to construct more families of WOM-codes. In fact, theconstruction in the previous section is a special case of this generalscheme.

Theorem 9.

Let C* be [m, t/2;q₁, . . . , q_(1/2) ] binary t/2-write WOM-code wheret is an even integer. For 1≦I≦t/2, let C_(i) be an [n,2;2^(nR)i,1,2^(nR)i,2] two-write WOM-code over GF(q_(i)), as constructedabove. Then, there exists an [nm, 2^(n)

^(1,1), 2^(n)

^(1,2), . . . , 2^(n)

^(1/21), 2^(n)

^(/2,2), t] binary t-write WOM-code of sum-rate

$\sum\limits_{i = 1}^{t/2}{\frac{R_{i,1} + R_{i,2}}{m}.}$Proof: For 1>i>t/2, let ξ_(i)*

_(i)* be the encoding, decoding maps on the i-th write of the WOM-codeC*, respectively. The definition of ξ_(i)*

_(i)* for 1>i>t/2, extends naturally to vectors by simply invoking themaps on each entry in the vector. Similarly, for 1≦i≦t/2, let us denoteby E_(i,1) and E_(i,2) the encoding maps of the first and second writes,and by D_(i,1) and D_(i,2) the decoding maps of the first and secondwrites of the WOM-code Ci, respectively. We will present thespecification of the encoding and decoding maps of the constructedt-write WOM-code.

In the following definitions of the encoding and decoding maps, weconsider the memory-state vector c to have n symbols of m bits each,i.e. cε(GF(2^(m)))^(n). For 1≦i≦t/2, the (2i−1)-st write and 2i-th writeare implemented as follows.

-   -   1. On the (2i−1)-st write, a message m₁ε{1, . . . , 2^(n)        ^(i,1)} is written to the memory-state vector c according to        ξ_(2i−1)(m ₁ ,c)=ξ_(i)*(ξ_(i,1)(m ₁),c).

The memory-state vector c is decoded according to

_(2i−1)(c)=

_(i,1)(

_(i)*(c)).

On the 2i-th write, a message m₂ε{1, . . . , 2^(nRi,1)} is writtenaccordingε_(2i)(m ₂)=ε_(i)*(ε_(i,2)(m ₂,

_(i)*(c)),c)

and the memory-state vector c is decoded according to

_(2i)(c)=

_(i,2)

_(i)*(c)).

We will demonstrate how this construction works in the followingexample.

Example 3

We choose a [3,3;4,3,2] three-write WOM-code as the code C*. This codeis depicted in FIG. 4 by a state diagram of an encoding map describingall three writes implanted in an encoder 10 to write to memory 12. Thethree-bit vector in each state is the memory-state and the number nextto it is the decoded value. We need to find three more two-writeWOM-codes over GF(4),GF(3), and GF(2). For the code C_(l) over

TABLE 1 SUM-RATES OF CONCATENATED WOM-CODES Number Achieved Maximum ofWrites New Rate New Rate 5 1.9689$\frac{{\log_{2}7} + {\log_{2}5} + 1}{3} = 2.0431$ 6 2.1331$\frac{{\log_{2}7} + {\log_{2}5} + {\log_{2}3}}{3} = 2.2381$ 7 2.1723$\frac{{\log_{2}7} + {\log_{2}5} + {\left( {{\log_{2}5} + 1} \right)/2}}{3} = 2.2634$8 2.2544$\frac{{\log_{2}7} + {\log_{2}5} + {\left( {{\log_{2}5} + {\log_{2}3}} \right)/2}}{3} = 2.3609$9 2.2918$\frac{{\log_{2}7} + {\log_{2}5} + {\left( {{\log_{2}7} + {\log_{2}5} + 1} \right)/3}}{3} = 2.3908$10  2.3466$\frac{{\log_{2}7} + {\log_{2}5} + {\left( {{\log_{2}7} + {\log_{2}5} + {\log_{2}3}} \right)/3}}{3} = 2.4588$GF(4), we ran a computer search to find a two-write WOM-code over GF(4)of sum-rate 2.6862. For the code C₂ over GF(3), we use the code withsum-rate 2.22 which we found above, and we use the binary two-writeWOM-code of sum-rate 1.4928 for the code C₃. Then, the sum-rate of thesix-write WOM-code is

$\frac{2.6793 + 2.22 + 1.49}{3} = {2.1297.}$

It is possible to construct a five-write WOM-code by writing a vector ofn bits in the last write so its sum-rate is

$\frac{2.6862 + 2.2205 + 1}{3} = {1.9689.}$

Note that if one of the codes in the general construction is binary thenwe can actually use a WOM-code that allows more than two writes. Thatis, in this construction we can use any binary multiple-write WOM-codeas the WOM-code C₃. Therefore, we can generate another family ofWOM-codes for t≧5. Their maximum achievable sum-rates are given by thefollowing formula

${\mathcal{R}_{t} = \frac{{\log_{2}7} + {\log_{2}5} + \mathcal{R}_{t - 4}}{3}},$for t≧5 and R_(r4) is the maximum achievable sum-rate for a (t−4)-writeWOM-code. Similarly, the constructed codes which we obtain using theWOM-codes found above have sum-rates

${\mathcal{R}_{t}^{\prime} = \frac{2.6862 + 2.2205 + \mathcal{R}_{t - 4}^{\prime}}{3}},$

where

_(t−4)′ is the best sum-rate of a constructed (t−4)-write WOM-code.Table IV summarizes these sum-rates.

Note that the construction is a special case of the generalizedconcatenated WOM-codes construction in which the WOM-code C* is chosento be a [2,2; 3, 2] binary two-write WOM-code.

The general method described in Theorem 7 provides us with many morefamilies of WOM-codes. However, in order to construct WOM-codes withhigh sum-rates, the WOM-code C* has to be chosen very carefully. Inparticular, it is important to choose such a WOM-code with as few cellsas possible, since the sum of all sum-rates of the non-binary two-writeWOM-codes is averaged over the number of cells of the WOM-code C*. Asthe number of short WOM-codes is small, there are only a small number ofpossibilities to check. However, our search for better WOM-codes withbetween six and ten writes using WOM-codes with few cells did not leadto any better results.

Fixed-Rate WOM-Codes

The WOM-code construction for more than two writes improved the achievedsum-rates only in the unrestricted-rate case. In this section, wepresent a method to construct fixed-rate WOM-codes. The method isrecursive and is based on the previously constructed unrestricted-rateWOM-codes.

Theorem 10.

Let C be an [n,t;2^(nR) ¹ ,2^(nR)2, . . . , 2^(nR) ¹ ] t-write WOM-code.Assume that for 1≦i≦t−1 there exists a fixed-rate WOM-code of sum-rateR_(i). Let R′₁, . . . , R′_(i) be a permutation of R₁, . . . ,R₁ suchthat R′₁≧ . . . , ≧R′₁. Then, there exists a fixed-rate t-write WOM-codeof sum-rate

$\frac{t \cdot \mathcal{R}_{1}^{\prime}}{1 + {\sum\limits_{i = 1}^{t - 1}\frac{{\mathbb{i}}\left( {\mathcal{R}_{t - i}^{\prime} - \mathcal{R}_{t - i + 1}^{\prime}} \right)}{R_{i}}}}.$

Proof: For simplicity, let us assume that R₁≧ . . . ≧R_(i) as it will beclear from the proof how to generalize to the arbitrary case. First, weadd (

_(i−1)−

_(t))_(n) more cells in order to write (

_(i−1)−

_(t))_(n) bits on the last write. This guarantees that the rates on thelast two writes are the same. Then, we add 2(R_(r2)−R_(t−1))n/R₂ morecells in order to write (R_(t)2−R_(r−1)) n more bits on each of the lasttwo writes. This part of the last two writes is invoked using thefixed-rate two-write WOM-code of sum-rate R₂ and therefore theadditional number of cells is 2(

_(t−2)−

_(t−1))n/R₂. This addition of cells guarantees that the rates on thelast three writes are all the same. In general, for 1≦i≦t−1 we add i(

_(t−i)−

_(t−i+1))n/R_(i) more cells such that (

_(t−i)−

_(t−i+1))^(n) more bits are written on each of the last i writes andtherefore the rates on the last i+1 writes are all the same. These bitsare written using the fixed-rate i-write WOM-code which is assumed toexist.

With the addition of these cells, the number of bits written on the i-thwrite for 1≦i≦t is

${{\mathcal{R}_{i}n} + {\sum\limits_{j = 1}^{i - 1}{\left( {\mathcal{R}_{j} - \mathcal{R}_{j + 1}} \right)n}}} = {\mathcal{R}_{1}{n.}}$Thus, the rates on all writes are the same and the generated WOM-code isfixed-rate.

The total number of bits we add is

${\sum\limits_{i = 1}^{t - 1}\frac{{{\mathbb{i}}\left( {\mathcal{R}_{t - i} - \mathcal{R}_{t - i + 1}} \right)}n}{R_{i}}},$and thus the sum-rate is

$\frac{{t \cdot \mathcal{R}_{1}}n}{n + {\sum\limits_{i = 1}^{t - 1}\frac{{{\mathbb{i}}\left( {\mathcal{R}_{t - i} - \mathcal{R}_{t - i + 1}} \right)}n}{R_{i}}}} = {\frac{t \cdot \mathcal{R}_{1}}{1 + {\sum\limits_{i = 1}^{i - 1}\frac{{\mathbb{i}}\left( {\mathcal{R}_{t - i} - \mathcal{R}_{t - i + 1}} \right)}{R_{i}}}}.}$

Let us demonstrate how to apply the last theorem. We start with thethree-write WOM-code. Its rates on the first, second, and third writesare 0.6291, 0.4811, 0.5, respectively. We add 0.0189n more cells inorder to guarantee that the rates on the last two writes are the same.Then we use the fixed-rate two-write WOM-code of sum-rate 1.4546. Hencewe add

$\frac{{2 \cdot \left( {0.6291 - 0.5} \right)}n}{1.4546} = {0.1775n}$more cells, yielding a fixed-rate three-write WOM-code of sum-rate

$\frac{3 \cdot 0.6291}{1.1964} = {1.5775.}$

If we used the best fixed-rate two-write WOM-code of sum-rate 1.546 andthe best three-write WOM-code of sum-rate 1.66, then we get a fixed-ratethree-write WOM-code of sum-rate 1.6263.

Note that we could use a two-write WOM-code such that 0.0189n bits arewritten on its first write and 0.1291n bits are written on its secondwrite. This will indeed add another small improvement to the sum-rate,however this scheme is not easy to generalize. Our goal here is to givea general method. We are aware that for each individual case it ispossible to use other unrestricted-rate WOM-codes that will provide aWOM-code of the desired sum-rate with slightly fewer cells.

Now we move to consider the four-write WOM-code. Its component rates are0.6291, 0.4811, 0.413, 1/3. We add three more groups of cells asfollows:

1) (0.413−1/3)n=0.0797n more cells, so that the last two write have thesame rate.

2) 2·(0.4811−0.413)n/1.4546=0.0936n more cells, so that the last threewrites have the same rate.

3) 3·(0.6291−0.4811)n/1.5731=0.2822n more cells, so that the last fourwrites have the same rate.

Then, we get a fixed-rate four-write WOM-code with sum-rate

$\frac{4 \cdot 0.6291}{1 + 0.0797 + 0.0936 + 0.2822} = {1.7298.}$

If we used the best fixed-rate two- and three-write WOM-codes and thebest unrestricted-rate four-write WOM-code, then we obtain a fixed-ratefour-write WOM-code of sum-rate 1.8249. Fixed-rate t-write WOM-code fort>4 can be similarly obtained. We summarize the results for thesum-rates that we actually found and the best ones we could find in thismethod in Table 2.

TABLE 2 SUM-RATES OF FIXED-RATE WOM-CODES Number Achieved Maximum ofWrites Sum-rate Sum-rate 3 1.5775 1.6263 4 1.7298 1.8249 5 1.8794 1.93026 1.9742 2.0570 7 1.991 2.0692 8 2.0375 2.1190 9 2.0951 2.1702 10 2.13272.2189

Tables 3 and 4 show a comparison of the sum-rates of unrestricted-rateand fixed-rate WOM-codes presented in this application and the bestpreviously known sum-rates for 2≦t≦10. The column labeled “Best Prior”is the highest sum-rate achieved by a previously reported t-writeWOM-code. The column “Achieved New Sum-rate” gives the sum-rates that weactually obtained through application of the new techniques. The column“Maximum New Sum-rate” lists the maximum possible sum-rates that can beobtained using our approach. Finally, the column “Upper Bound” gives themaximum possible sum-rates for t-write WOM-codes.

For unrestricted-rate two-write WOM-codes, the results were found by thecomputer search method. For three and four writes, we used the WOM-codesdescribed for multiple writes, and for 5≦t≦10, we used the WOM-codesdiscussed for concatenated code. For fixed-rate two-write WOM-codes, weagain used the computer search method of this Section providing twowrite codes. The constructions for more than two writes were obtained byapplication of Theorem 10.

TABLE 3 COMPARISON WITH KNOWN UNRESTRICTED-RATE WOM-CODES Number BestAchieved Maximum Upper of Writes Prior New Sum-rate New Sum-rate Bound 21.3707 1.4928 1.585 1.585 3 1.5302 1.6102 1.661 2 4 1.7524 1.8566 1.95342.3219 5 1.7524 1.9689 2.0431 2.585 6 1.7524 2.1331 2.2381 2.8074 71.8232 2.1723 2.2634 3 8 1.8824 2.2544 2.3609 3.1699 9 1.9535 2.29182.3908 3.3219 10 2.0144 2.3466 2.4588 3.4594

TABLE 4 COMPARISON WITH KNOWN FIXED-RATE WOM-CODES Number Best AchievedMaximum Upper of Writes Prior New Sum-rate New Sum-rate Bound 2 1.3431.4546 1.546 1.546 3 1.4348 1.5775 1.6263 1.9366 4 1.6042 1.7298 1.82492.2436 5 1.6279 1.8794 1.9302 2.4965 6 1.7143 1.9742 2.0570 2.7120 71.8232 1.991 2.0692 2.9001 8 1.8824 2.0375 2.1190 3.0664 9 1.9535 2.09512.1702 3.2157 10 2.0144 2.1327 2.2189 3.3520

REFERENCES

-   [1] D. Blackwell, “Statistics 262,” Course taught at the University    of California, Berkeley, Spring 1963.-   [2] T. Berger, F. Jelinek, and J. K. Wolf, “Permutation codes for    sources,” IEEE Trans. Inform. Theory, vol. 18, no. 1, pp. 160-168,    January 1972.-   [3] R. Brent, S. Gao, and A. Lauder, “Random Krylov spaces over    finite fields,” SIAM J. Discrete Math, vol. 16, no. 2, pp. 276-287,    February 2003.-   [4] P. Cappelletti, C. Golla, P. Olivo, and E. Zanoni, Flash    Memories, Boston: Kluwer Academic, 1999.-   [5] G. D. Cohen, P. Godlewski, and F. Merkx, “Linear binary code for    writeonce memories,” IEEE Trans. Inform. Theory, vol. 32, no. 5, pp.    697- 700, October 1986.-   [6] J. H. Conway and N. J. Sloane, “Orbit and coset analysis of the    Golay and related codes,” IEEE Trans. Inform. Theory, vol. 36, no.    5, pp. 1.038-1050, September 1990.-   [7] T. M. Cover, “Enumerative source encoding,” IEEE Trans. Inform.    Theory, vol. 19, no. 1, pp 73-77, January 1973.-   [8] A. Fiat and A. Shamir, “Generalized “write-once” memories,” IEEE    Trans. Inform. Theory, vol. 30, no. 3, pp. 470-480, September 1984.-   [9] F. Fu and A. J. Han Vinck, “On the capacity of generalized    write-once memory with state transitions described by an arbitrary    directed acyclic graph,” IEEE Trans. Inform. Theory, vol. 45, no. 1,    pp. 308-313, September 1999.-   [10] E. Gal and. S. Toledo, “Algorithms and data structures for    flash memories,” ACM Computing Surveys, vol. 37, pp. 138-163, June    2005.-   [11] S. I. Gel'fand, “Capacity of one broadcast channel,” Problemy    Peredachi Informatsii, vol. 13, no. (3), pp. 106-108, 1977.-   [12] P. Godlewski, “WOM-codes construits a partir des codes de    Hamming,” Discrete Math., vol. 65, no. 3, pp. 237-243, July 1987.-   [13] L. Grupp, A. Caulfield, J. Coburn, S. Swanson, E.    Yaakobi, P. H. Siegel, and J. K. Wolf, “Characterizing flash memory:    anomalies, observations, and applications,” MICRO-42, pp. 24-33,    December 2009.-   [14] c. Heegard, “On the capacity of pelinanent memory,” IEEE Trans.    Inform. Theory, vol. 31, no. 1, pp. 34-42, January 1985.-   [15] A. Jiang, “On the generalization of error-correcting WOM    codes,” Proc. IEEE Int. Symp. Inform. Theory, pp. 1391-1395, Nice,    France, June 2007.-   [16] A. Jiang, V. Bohossian, and J. Bruck, “Floating codes for joint    information storage in write asymmetric memories,” Proc. IEEE Int.    Symp. Inform. Theory, pp. 1166-1170, Nice, France, June 2007.-   [17] A. Jiang and J. Bruck, “Joint coding for flash memory storage,”    Proc. IEEE Int. Symp. Inform. Theory, pp. 1741-1745, Toronto,    Canada, July 2008.-   [18] S. Kayser, E. Yaakobi, P. H. Siegel, A. Vardy, and J. K. Wolf,    “Multiplewrite WOM-codes,” Proc. 48-th Annual Allerton Conference on    Communication, Control and Computing, Monticello, Ill.,    September 2010. [19] D. E. Knuth, “Efficient balanced codes,” IEEE    Trans. Inform. Theory, vol. 32, no. 1, pp. 51-53, January 1986.-   [20] H. Mandavifar, P. H. Siegel, A. Vardy, J. K. Wolf, and E.    Yaakobi, “A nearly optimal construction of flash codes,” Proc. IEEE    Int. Symp. Inform. Theory, pp. 1239-1243, Seoul, Korea, July 2009.-   [21] F. Merkx, “Womcodes constructed with projective geometries,”    Traitement du signal, vol. 1, no. 2-2, pp. 227-231, 1984.-   [22] R. L. Rivest and A. Shamir, “How to reuse a write-once memory,”    Inform. and Contr., vol. 55, no. 1-3, pp. 1-19, December 1982.-   [23] R. M. Roth, February 2010, personal communication.-   [24] R. M. Roth, Introduction to Coding Theory, Cambridge University    Press, 2005.-   [25] J. P. M. Schalkwijk, “An algorithm for source coding,” IEEE    Trans. Inform. Theory, vol. 18, no. 3, pp. 395-399, May 1972.-   [26] c. Tian, V A. Vaishampayan, and N. J. A. Sloane, “Constant    Weight codes: a geometric approach based on dissections,”,    CiteSeerX—Scientific Literature Digital Library and Search Engine,    2010.-   [27] J. K. Wolf, A. D. Wyner, J. Ziv, and J. Korner, “Coding for a    write-once memory,” AT&T Bell Labs. Tech. J., vol. 63, no. 6, pp.    1089-1112, 1984.-   [28] Y. Wu, “Low complexity codes for writing write-once memory    twice,” Proc. IEEE Int. Symp. Inform. Theory, pp. 1928-1932, Austin,    Tex., June 2010.-   [29] Y. Wu and A. Jiang, “Position modulation code for rewriting    write-once memories,” acceptted by IEEE Trans. Inform. Theory,    October 2010.-   [30] E. Yaakobi, S. Kayser, P. H. Siegel, A. Vardy, and J. K. Wolf,    “Efficient two-write WOM-codes,” Proc. IEEE Inform. Theory Workshop,    Dublin, Ireland, August 2010.-   [31] E. Yaakobi, J. Ma, L. Grupp, P. H. Siegel, S. Swanson,    and J. K. Wolf, “Error characterization and coding schemes for flash    memories,” Proc. Workshop on the Application of Communication Theory    to Emerging Memory Technologies, Miami, Fla., December 2010.-   [32] E. Yaakobi, P. H. Siegel, A. Vardy, and J. K. Wolf, “Multiple    errorconecting WOM-codes,” Proc. IEEE Int. Symp. Inform. Theory, pp.    1933-1937, Austin, Tex., June 2010.-   [33] G. Zernor, “Problemes combinatoires lies it I'ecriture sur des    mernoires,” Ph.D. Dissertation, ENST, Paris, France, November 1989.-   [34] G. Zemor and G. D. Cohen, “Error-correcting WOM-codes,” IEEE    Trans. Inform. Theory, vol. 37, no. 3, pp. 730-734, May 1991.

While specific embodiments of the present invention have been shown anddescribed, it should be understood that other modifications,substitutions and alternatives are apparent to one of ordinary skill inthe art. Such modifications, substitutions and alternatives can be madewithout departing from the spirit and scope of the invention, whichshould be determined from the appended claims.

Various features of the invention are set forth in the appended claims.

The invention claimed is:
 1. A method for writing data to a nontransient medium using WOM-codes to provide a guaranteed number ofmultiple writes, the method comprising steps of: defining, in anelectronic encoder, a parity check matrix H_(v) for a linear code C[n,k]to code the data such that the i-th column of H_(v), 1≦i≦n, is the i-thcolumn of H if v_(i)=0 and otherwise it is a zeros column, wherein {v₁,v₂, . . . v, _(|Vc|)} is an ordering of a set of memory vectors Vc;defining, in the encoder, a set Vc of memory vectors s₁ such that eachvector v has a rank as follows:V _(c) ={vε{0,1}^(n)|rank(

_(v))=n−k} writing, by the encoder, a symbol to the non transient mediumover an alphabet of size |Vc|.
 2. The method of claim 1, furthercomprising: on a second write, writing, by the encoder, a second vectors₂ of n−k bits to the non transient medium, wherein v₁ is the programmedvector on the first write, s₁=H_(v1), then E₂(s₂,v_(i))=v₁+V₂, and v₂ isa solution of the equation H_(v)·v₂=s₁+s₂.
 3. A method for decoding dataencoded on a transient medium according to claim 2, comprising readingdata from the transient medium and applying, by an electronic decoder, adecoding map such that D₂, if c is the vector of programmed cells of thetransient medium, then the decoded value of the n−k bits is given byD₂(c)=H·c=H·v₁+H·v₂=s₁+s₁+s₂=s₂.
 4. The method of claim 1, applied to alinear code, such that the parity check matrix H_(v) is selecteduniformly at random and an indicator random variable X_(v) (H) on thespace of all matrices as follows ${X_{v}(H)} = \left\{ {\begin{matrix}1 & {{{if}\mspace{14mu} v} \in V_{C}} \\0 & {otherwise}\end{matrix}.} \right.$
 5. The method of claim 1, wherein the nontransient medium comprises a memory having cells that can change statefrom zero to one but not from one to zero except upon an erase of theentire memory, and said writing writes the symbol into cells without theerase of the entire memory.
 6. The method of claim 1, wherein theguaranteed number of writes is three, and said writing data comprises 1)on a first write, a message m from a set {1, . . . , 2^(nR) ¹ } iswritten in 2 n cells:ε₁(m)=φ(ε_(3,1)(m)), 2) on a second write, a message m from the set {1,. . . , 2^(nR) ² } is written in the 2n cells, wherein c is a programmedvector on the first write, whereε₂(m,c)=φ(ε_(3,2)(m,φ ⁻¹(c))), such that first the memory-state vector cis converted to a ternary vector and then the ternary vector is encodedusing the encoding E_(3,2) and the new message, producing a new ternarymemory-state vector, and subsequently the last vector is converted to a2n-bit vector; and 3) on a third write, an n-bit vector v is written,with c=((c_(1,1),c_(1,2)), . . . , (c_(n,1),c_(n,2))) being the currentmemory-state vector, and then,E ₃(v,c)=((c′ _(1,1) ,c′ _(1,2)), . . . ,(c∝ _(n,1) ,c′ _(n,2))) is avector, defined as follows: for 1≦i≦n, (c′_(i,1),c′_(i,2))=(1,1) ifv_(i)=1 and otherwise (c′_(i,1),c′_(i,2)).
 7. A method for writing datato a non transient medium using WOM-codes to provide more than twowrites, the method comprising: defining, in an electronic encoder,parity-check matrix H of size (n−k)×n, wherein for a vector v of lengthn over GF(q), let H(v) be the matrix H with zero columns replacing thecolumns that correspond to the positions of the non-zero values in v,and a set of memory vectors V_(c) ^((q)) is defined such thatV _(c) ^((q)) ={vε(GF(q))^(n)|rank(

(v))=n−k} defining, in the encoder, a non binary two-write codeaccording to claim 1, wherein [n,2;|V_(c) ^((q))|,q^(n−k)]; writingdata, by the encoder, to the non transient medium more than two timesover the alphabet of V_(c) ^((q)).
 8. The method of claim 7, whereinsaid writing writes the symbol into cells without the erase of theentire memory.
 9. A method for writing data to a non transient mediumhaving q levels, where q is a prime number or a power of a prime numbergreater than 2, using WOM-codes to provide a guaranteed number ofmultiple writes, the method comprising steps of: defining, in anelectronic encoder, a parity check matrix H for a linear code C[n,k]over GF(q) with a parity-check matrix H of size (n−k)×n to code the datasuch that the i-th column of H_(v), 1≦i≦n, is the i-th column of H ifv_(i)=0 and otherwise it is a zeros column, wherein {v₁, v₂, . . . v,_(|Vc|)} is an ordering of a set of memory vectors Vc; defining, in theencoder, a set Vc of memory vectors Si such that each vector v has arank as follows:V _(c) ^((q)) ={vε(GF(q))^(n)|rank(H(v))=n−k} writing, by the encoder, asymbol to the non transient medium over an alphabet of size |Vc|. 10.The method of claim 9, wherein said writing writes the symbol into cellswithout the erase of the entire memory.
 11. The method of claim 9,wherein the linear code C[n,k] comprises an unrestricted linear code.